Overview

This document provides an overview of the univariate relationships of all covariates with the absolute parameter deviation. We separate the relationships by focussing on one method as the target method and then investigating the relationships for each of the remaining methods with this method.

DV: Absolute Deviation from Complete Pooling MLE Estimate

We begin by investigating the abolsute relationship from the simplest method, the complete pooling MLE method (i.e., y always refers to Comp MLE" and x refers to the other method in the pair). This leaves us with 13921 observations for the analysis.

Effect of Method

We can also look at the histogram of the absolute deviation across methods.

Effects of Continuous Covariates

In the following plots, the blue line shows the fitted model (in case it is not a simple linear relationship, the transformation of the independent variable is given in parentheses in the x-axis label). The \(R^2\) value shon in the plot is the \(R^2\) of this model (i.e., the blue line). The red line shows a GAM on the independent variable with shrinkage applied thin plagte regression spline.

In case observations had to be removed for the analysis, the percentage of removed (rem) observations is also shown in the x-axis caption.

Effect of Parameter Estimate

Standard Error

Hetereogeneity

The data suggests a step-like relationship such that only values that are at or near zero show a considerable probability of non-zero absolute deviations. To look at this further, we can see how probable it is to observe values near zero. The following table shows that at least 80% of observations have a log1p value that is very near to zero.

## # A tibble: 8 x 3
##   cond_x     less_than_00001 less_than_01
##   <fct>                <dbl>        <dbl>
## 1 Comp Bayes           0.820        0.912
## 2 No asy               0.820        0.912
## 3 No PB                0.820        0.912
## 4 No NPB               0.834        0.926
## 5 No Bayes             0.814        0.909
## 6 Beta PP              0.818        0.911
## 7 Trait_u PP           0.870        0.921
## 8 Trait PP             0.816        0.906

If we look at the conditionmal disttirbution of absolute deviation whether or not it is very near to zero, we can see that there is some evidence for the step-like relationship, but the pattern is not overwhelming.

Rho

Fungibility

Model Fit

Relative Parameter Information

Effects of Categroical Covariates

The effect of covariate is shown in two ways. The table below all plots gives the \(R^2\) values for the model given the covariate across all comparison methods. In case the number of levels is not too large, a plot of the difference in absolute deviation conditional on the factor levels is shown. Some factor levels may be removed for plotting (i.e., those levels for which the proportion of observations is less than 0.04). In this case, the number of removed levels is also given.

Meta Data

## # A tibble: 4 x 10
##   covariate nlevels `Beta PP` `Comp Bayes` `No asy` `No Bayes` `No NPB` `No PB` `Trait PP` `Trait_u PP`
##   <chr>       <int>     <dbl>        <dbl>    <dbl>      <dbl>    <dbl>   <dbl>      <dbl>        <dbl>
## 1 model           9     0.217       0.0706    0.210     0.0852    0.132   0.107      0.283        0.252
## 2 model2         13     0.230       0.0720    0.234     0.112     0.152   0.129      0.292        0.270
## 3 parameter      53     0.402       0.120     0.536     0.351     0.358   0.369      0.488        0.453
## 4 dataset       166     0.328       0.655     0.301     0.285     0.219   0.199      0.334        0.343

Categorical Covariates

## # A tibble: 2 x 10
##   covariate  nlevels `Beta PP` `Comp Bayes` `No asy` `No Bayes` `No NPB` `No PB` `Trait PP` `Trait_u PP`
##   <chr>        <int>     <dbl>        <dbl>    <dbl>      <dbl>    <dbl>   <dbl>      <dbl>        <dbl>
## 1 population       5   0.115        0.0293   0.124      0.0227   0.0818  0.0572     0.153        0.122  
## 2 sci_goal         2   0.00127      0.00151  0.00445    0.00643  0.00428 0.00443    0.00140      0.00120

DV: Absolute Deviation from Latent Trait Partial Pooling Estimate

In the second analysis, we focus on investigating the absolute deviation from the most complex method, the latent trait partial pooling method (i.e., y always refers to Trait PP and x refers to the other method in the pair). This leaves us with 13184 observations for the analysis.

Effect of Method

We can also look at the histogram of the absolute deviation across methods.

Effects of Continuous Covariates

In the following plots, the blue line shows the fitted model (in case it is not a simple linear relationship, the transformation of the independent variable is given in parentheses in the x-axis label). The \(R^2\) value shon in the plot is the \(R^2\) of this model (i.e., the blue line). The red line shows a GAM on the independent variable with shrinkage applied cubic regression spline.

In case observations had to be removed for the analysis, the percentage of removed (rem) observations is also shown in the x-axis caption.

Effect of Parameter Estimate

Standard Error

Hetereogeneity

The data suggests a step-like relationship such that only values that are at or near zero show a considerable probability of non-zero absolute deviations. To look at this further, we can see how probable it is to observe values near zero. The following table shows that at least 80% of observations have a log1p value that is very near to zero.

## # A tibble: 8 x 3
##   cond_x     less_than_00001 less_than_01
##   <fct>                <dbl>        <dbl>
## 1 Comp MLE             0.816        0.906
## 2 Comp Bayes           0.816        0.906
## 3 No asy               0.816        0.906
## 4 No PB                0.816        0.907
## 5 No NPB               0.830        0.921
## 6 No Bayes             0.809        0.903
## 7 Beta PP              0.814        0.906
## 8 Trait_u PP           0.867        0.916

If we look at the conditionmal disttirbution of absolute deviation whether or not it is very near to zero, we can see that there is some evidence for the step-like relationship, but the pattern is not overwhelming.

Rho

Fungibility

Model Fit

Relative Parameter Information

Effects of Categroical Covariates

The effect of covariate is shown in two ways. The table below all plots gives the \(R^2\) values for the model given the covariate across all comparison methods. In case the number of levels is not too large, a plot of the difference in absolute deviation conditional on the factor levels is shown. Some factor levels may be removed for plotting (i.e., those levels for which the proportion of observations is less than 0.04). In this case, the number of removed levels is also given.

Meta Data

## # A tibble: 4 x 10
##   covariate nlevels `Beta PP` `Comp Bayes` `Comp MLE` `No asy` `No Bayes` `No NPB` `No PB` `Trait_u PP`
##   <chr>       <int>     <dbl>        <dbl>      <dbl>    <dbl>      <dbl>    <dbl>   <dbl>        <dbl>
## 1 model           9    0.0925        0.140      0.283    0.175      0.166    0.108  0.0915        0.138
## 2 model2         13    0.118         0.143      0.292    0.189      0.212    0.120  0.104         0.177
## 3 parameter      53    0.305         0.195      0.488    0.434      0.423    0.356  0.380         0.351
## 4 dataset       157    0.299         0.664      0.334    0.289      0.411    0.211  0.196         0.314

Categorical Covariates

## # A tibble: 2 x 10
##   covariate  nlevels `Beta PP` `Comp Bayes` `Comp MLE` `No asy` `No Bayes` `No NPB` `No PB` `Trait_u PP`
##   <chr>        <int>     <dbl>        <dbl>      <dbl>    <dbl>      <dbl>    <dbl>   <dbl>        <dbl>
## 1 population       5 0.0678        0.0287      0.153   0.0913      0.0336  0.0373   0.0286       0.118  
## 2 sci_goal         2 0.0000375     0.000636    0.00140 0.000775    0.00645 0.000907 0.00129      0.00151